A comparative analysis of early and late fusion for the multimodal two-class problem

In this article we carry out a comparison between early (feature) and late (score) multimodal fusion, for the two-class problem. The is made first from general perspective, then specific mathematical analysis. Thus, deduce error probability expressions uncorrelated correlated multivariate Gaussian distribution, assuming perfect model knowledge (Bayes rates). We also corresponding when to be learned finite training set, demonstrating its convergence Bayes rates as set size goes infinite. These demonstrates that fusion best option with knowledge, both degrades due set. This degradation showed grater dimensionality increase of feature space, so, eventually, could better in practical setting. analysis suggests convenience using a, so called, convergence factor , quantify if appropriate close enough rate. Different simulated experiments have been verify validity analysis, well possible extension non-Gaussian models.